www.econstor.eu GENERALIZED STOCHASTIC GRADIENT LEARNING
نویسندگان
چکیده
We study the properties of generalized stochastic gradient (GSG) learning in forward-looking models. We examine how the conditions for stability of standard stochastic gradient (SG) learning both differ from and are related to E-stability, which governs stability under least squares learning. SG algorithms are sensitive to units of measurement and we show that there is a transformation of variables for which E-stability governs SG stability. GSG algorithms with constant gain have a deeper justification in terms of parameter drift, robustness and risk sensitivity. JEL Code: C62, C65, D83, E10, E17.
منابع مشابه
Generalized Stochastic Gradient Learning∗
We study the properties of the generalized stochastic gradient (GSG) learning in forward-looking models. GSG algorithms are a natural and convenient way to model learning when agents allow for parameter drift or robustness to parameter uncertainty in their beliefs. The conditions for convergence of GSG learning to a rational expectations equilibrium are distinct from but related to the well-kno...
متن کاملADAPTIVITY OF AVERAGED STOCHASTIC GRADIENT DESCENT Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression
In this paper, we consider supervised learning problems such as logistic regression and study the stochastic gradient method with averaging, in the usual stochastic approximation setting where observations are used only once. We show that after N iterations, with a constant step-size proportional to 1/R √ N where N is the number of observations and R is the maximum norm of the observations, the...
متن کاملIdentification of Multiple Input-multiple Output Non-linear System Cement Rotary Kiln using Stochastic Gradient-based Rough-neural Network
Because of the existing interactions among the variables of a multiple input-multiple output (MIMO) nonlinear system, its identification is a difficult task, particularly in the presence of uncertainties. Cement rotary kiln (CRK) is a MIMO nonlinear system in the cement factory with a complicated mechanism and uncertain disturbances. The identification of CRK is very important for different pur...
متن کاملStatistical analysis of stochastic gradient methods for generalized linear models
We study the statistical properties of stochastic gradient descent (SGD) using explicit and implicit updates for fitting generalized linear models (GLMs). Initially, we develop a computationally efficient algorithm to implement implicit SGD learning of GLMs. Next, we obtain exact formulas for the bias and variance of both updates which leads to two important observations on their comparative st...
متن کاملTime scales of adaptive behavior and motor learning in the presence of stochastic perturbations.
In this paper, the major assumptions of influential approaches to the structure of variability in practice conditions are discussed from the perspective of a generalized evolving attractor landscape model of motor learning. The efficacy of the practice condition effects is considered in relation to the theoretical influence of stochastic perturbations in models of gradient descent learning of m...
متن کامل